Video coding techniques may use prediction based coding in order to be efficient. Data at a given frame is predicted, on a block basis, from already decoded data, which can either be from other reference frames (i.e., “inter” prediction), or from the already decoded data at the same frame (i.e., “intra” prediction). The residual error, generated after prediction is subtracted from the original data, is then typically transformed, quantized, and encoded. The type of prediction used at a given spatial location of a given frame is adaptively selected such that final coding is as efficient as possible. This selection relies on the optimization of a rate-distortion measure. Indeed, the predictor leading to the lowest distortion with the lowest bitrate is typically selected among all possible prediction modes.
In some cases, the best predictor in terms of rate distortion may not give accurate predicted data, generating, then, a high amount of residual error that has to be coded. Inaccuracy may be due to the rate constraint which leads the predictor selection to a compromise between bitrate cost and distortion or simply because available prediction models aren't appropriate. Intra prediction in the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding (AVC) standard/international Telecommunication Union, Telecommunication Sector (ITU-T) H.264 recommendation (hereinafter the “MPEG-4 AVC standard” is an example where a low-pass operator is used to predict the data in a given block using information from its decoded neighboring blocks. However, such predictors are unable to handle high frequencies and textured data.
In some state of the art video encoders/decoders such as, for example, those in compliance with the MPEG-4 AVC Standard, prediction refinement by the use of the so-called “deblocking filter” is utilized within the coding/decoding process. Coding inaccuracies, introduced by transform-based coding of the residual error, may be reduced by means of a filter that operates on reconstructed frames as a last step in the coding loop. Other in-loop filters have been proposed in order to overcome the limitations of the MPEG-4 AVC Standard deblocking filter. Typically, these filters are applied on the reconstructed pictures.
In-loop filtering after reconstruction allows for the recovery of part of the information lost during the quantization step in error residual coding. However, it is not expected to help reduce the amount of information to be encoded in the current picture, as it is applied on reconstructed images. In order to reduce the amount of information to be encoded, the prediction signal can be improved. Traditionally, this has been done by the inclusion of increasingly more sophisticated prediction modes.
Algorithms for the estimation of missing data, some of which may be referred to as “inpainting” algorithms, may be based on, for example, diffusion principles and/or texture growing, or nonlinear sparse decompositions de-noising. These algorithms may try to estimate the values of missing data based on known neighboring data. Indeed, one could imagine having a missing block within a picture, and recovering the missing block by estimating the missing block from the data available in some neighboring block. These algorithms generally assume there is no knowledge about the data missing, i.e., they only rely upon the neighboring available data to estimate the missing data.
Turning to FIG. 1, a video encoder capable of performing video encoding in accordance with the MPEG-4 AVC standard is indicated generally by the reference numeral 100.
The video encoder 100 includes a frame ordering buffer 110 having an output in signal communication with a non-inverting input of a combiner 185. An output of the combiner 185 is connected in signal communication with a first input of a transformer and quantizer 125. An output of the transformer and quantizer 125 is connected in signal communication with a first input of an entropy coder 145 and a first input of an inverse transformer and inverse quantizer 150. An output of the entropy coder 145 is connected in signal communication with a first non-inverting input of a combiner 190. An output of the combiner 190 is connected in signal communication with a first input of an output buffer 135.
A first output of an encoder controller 105 is connected in signal communication with a second input of the frame ordering buffer 110, a second input of the inverse transformer and inverse quantizer 150, an input of a picture-type decision module 115, an input of a macroblock-type (MB-type) decision module 120, a second input of an intra prediction module 160, a second input of a deblocking filter 165, a first input of a motion compensator 170, a first input of a motion estimator 175, and a second input of a reference picture buffer 180.
A second output of the encoder controller 105 is connected in signal communication with a first input of a Supplemental Enhancement Information (SEI) inserter 130, a second input of the transformer and quantizer 125, a second input of the entropy coder 145, a second input of the output buffer 135, and an input of the Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) inserter 140.
A first output of the picture-type decision module 115 is connected in signal communication with a third input of a frame ordering buffer 110. A second output of the picture-type decision module 115 is connected in signal communication with a second input of a macroblock-type decision module 120.
An output of the Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) inserter 140 is connected in signal communication with a third non-inverting input of the combiner 190.
An output of the inverse quantizer and inverse transformer 150 is connected in signal communication with a first non-inverting input of a combiner 119. An output of the combiner 119 is connected in signal communication with a first input of the intra prediction module 160 and a first input of the deblocking filter 165. An output of the deblocking filter 165 is connected in signal communication with a first input of a reference picture buffer 180. An output of the reference picture buffer 180 is connected in signal communication with a second input of the motion estimator 175. A first output of the motion estimator 175 is connected in signal communication with a second input of the motion compensator 170. An output of reference picture buffer 180 is connected in signal communication with a third input of the motion compensator 170. A second output of the motion estimator 175 is connected in signal communication with a third input of the entropy coder 145.
An output of the motion compensator 170 is connected in signal communication with a first input of a switch 197. An output of the intra prediction module 160 is connected in signal communication with a second input of the switch 197. An output of the macroblock-type decision module 120 is connected in signal communication with a third input of the switch 197. The third input of the switch 197 determines whether or not the “data” input of the switch (as compared to the control input, i.e., the third input) is to be provided by the motion compensator 170 or the intra prediction module 160. The output of the switch 197 is connected in signal communication with a second non-inverting input of the combiner 119 and with an inverting input of the combiner 185.
Inputs of the frame ordering buffer 110 and the encoder controller 105 are available as input of the encoder 100, for receiving an input picture 101. Moreover, an input of the Supplemental Enhancement Information (SEI) inserter 130 is available as an input of the encoder 100, for receiving metadata. An output of the output buffer 135 is available as an output of the encoder 100, for outputting a bitstream.
Turning to FIG. 2, a video decoder capable of performing video decoding in accordance with the MPEG-4 AVC standard is indicated generally by the reference numeral 200.
The video decoder 200 includes an input buffer 210 having an output connected in signal communication with a first input of the entropy decoder 245. A first output of the entropy decoder 245 is connected in signal communication with a first input of an inverse transformer and inverse quantizer 250. An output of the inverse transformer and inverse quantizer 250 is connected in signal communication with a second non-inverting input of a combiner 225. An output of the combiner 225 is connected in signal communication with a second input of a deblocking filter 265 and a first input of an intra prediction module 260. A second output of the deblocking filter 265 is connected in signal communication with a first input of a reference picture buffer 280. An output of the reference picture buffer 280 is connected in signal communication with a second input of a motion compensator 270.
A second output of the entropy decoder 245 is connected in signal communication with a third input of the motion compensator 270 and a first input of the deblocking filter 265. A third output of the entropy decoder 245 is connected in signal communication with an input a decoder controller 205. A first output of the decoder controller 205 is connected in signal communication with a second input of the entropy decoder 245. A second output of the decoder controller 205 is connected in signal communication with a second input of the inverse transformer and inverse quantizer 250. A third output of the decoder controller 205 is connected in signal communication with a third input of the deblocking filter 265. A fourth output of the decoder controller 205 is connected in signal communication with a second input of the intra prediction module 260, with a first input of the motion compensator 270, and with a second input of the reference picture buffer 280.
An output of the motion compensator 270 is connected in signal communication with a first input of a switch 297. An output of the intra prediction module 260 is connected in signal communication with a second input of the switch 297. An output of the switch 297 is connected in signal communication with a first non-inverting input of the combiner 225.
An input of the input buffer 210 is available as an input of the decoder 200, for receiving an input bitstream. A first output of the deblocking filter 265 is available as an output of the decoder 200, for outputting an output picture.